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SUMMARY  .  ^ 

Our  study  has'shown  that  using  \KC%  2  to  estimate^ and  R  for  small  sample  sizes, 
produces  estimates  that  are  too  conservative  when  compared  to  Bartholomew's  exact 
method.  If  v^is  used  to  estimate  ^and  /ft,  optimistic  estimates  Will  be  obtained. 
Better  estimates  can  be  obtained  for  small  sample  sizes  by  using  \i2r  +j-  Therefore,  for 

of  J  ■  j.  j  t  2*  *  ‘  *>  t 

small  sample  sizes,  we  recommend  using  \  2c  +  1  instead  of  \  2f  +  2  or  W 

'  >  A  f  1 

During  our  ''study,  much  to  our  surprise,  we^found  that  the  lower  confidence  limits 
could  not  be  computed  for  all  realistic  values  of\|)in  Bartholomew’s  exact  method.  This 
occurred  when  ^approached  its  maximum  theoretical  value  for  very  small  sample  sizes. 
An  explanation  for  this  has  not  yet  been  determined  by  the  authors. 
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I.  INTRODUCTION 


The  problem  of  estimating  reliability  using  censored  data  and  determining  a  lower 
confidence  limit  oil  reliability  is  discussed  in  this  paper.  Censoring  occurs  when  testing 
is  terminated  before  all  items  have  failed.  There  are  two  basic  methods  of  censoring: 
(1)  Type  1  censoring,  where  testing  is  terminated  at  some  predetermined  time:  and  I 2 ) 
Type  II  censoring,  where  testing  is  terminated  after  a  preassigned  number  of  failures. 
When  failures  occur,  either  an  item  is  replaced  or  it  is  not  replaced.  For  this  paper,  we 
are  interested  in  Type  I  censoring  without  replacement  for  small  sample  sizes  (less  than 
25)  with  one  or  more  failures.  Suppose  we  have  n  items  on  test  until  time  T  (censored 
time).  Each  item  is  tested  until  it  fails  or  the  c  failures  occur  at  times,  t,.  to.  •  •  •  f, 
where  t;  <  t;  +  t  <  T. 

One  could  simply  ignore  the  failure  times  and  just  consider  that  c  failures  occurred 
out  of  n  items,  then  use  the  binomial  distribution  to  put  a  lower  confidence  limit  (l.c.l.) 
on  reliability.  However,  too  much  information  is  ignored  and  the  confidence  bound  is 
very  conservative. 

A  better  approach  would  be  to  utilize  both  number  of  failures  and  failure  times.  If 

the  failure  times  (t|)  are  assumed  to  be  exponentially  distributed,  the  probability  density 

-  —  "  -  1  ' 

function  (pdf)  for  t  is  f(t)  =  -i-  e  9  ,  and  the  reliability  is  R(t)  —  e  6 ,  t  >  0.  0  >  0. 

0 

In  a  1960  paper.  Epstein1  gave  estimation  procedures  for  both  point  and  confidence  limit 
estimates  for  0.  For  Type  I  censoring  without  replacement  of  the  failed  items,  the 
exponential  parameter  0  can  be  estimated  by  its  maximum  likelihood  estimate,  (m.l.<\ ). 
which  is 

0  =  A/c  (1) 

where 

r 

V]  t;  +  (n  -  c)  T  =  Total  amount  of  test  time 

i=i 

sum  of  the  failure  times  for  the  c  items  that  failed 

total  number  of  items  on  test 
c  =  total  number  of  failures 
T  =  censored  time 

(This  is  a  biased  estimate:  E(0)  =  l/c  [ E(t;)  +  (n  c)  Tj 

i  =  l 

=  l/c  [c0  +  (n  c)  Tj=  0  +  — — — T) 

C 

f'4  t  Bpiijamiij  *  -miation  from  Info  T»v-t  Data*  Technometnr?.  Vol  -  No  4  Nov  19f.fi 


A  = 


i— i 

n  = 


v 


Then  the  lower  100  (1  -  a  )  percent  confidence  interval  for  9  is  estimated  by 

^c  +  2  =  --V\?2c  +  2(a)  ('-) 

where  \22c  +  2(Q)  *s  l^e  upper  a  percentage  point  of  a  \2  distribution  with  2c  +  2 


degrees  of  freedom.  Therefore,  the  lower  limit  on  reliability  using  this  l.c.l.  for  0  would 

t 


be  Roc  +  2^)  =  e  Epstein's  approximate  procedure  is  the  most  commonly  used 

and  is  cited  in  text  books  such  as  Bazovsky2;  Mann,  Shafer,  and  Singpurwalla'5;  Hahn 
and  Shapiro4;  and  others. 


In  lieu  of  the  many  approximations  that  exist  for  the  l.c.l.  of  9,  the  exact  distribu¬ 
tion  of  9  can  be  found.  This  exact  distribution  of  9  was  derived  by  D.J.  Bartholomew’’ 
for  small  sample  sizes  where  at  least  one  failure  occurs,  i.e..  c  >  0. 


Bartholomew's  exact  method  of  calculating  the  l.c.l.,  0exart.  for  the  exponential 
parameter.  0.  is  the  solution  to 


Pr  (0O  >0)  = 


1  _  e-nT/('^c  c==1 


V  I  "  )  Els)  H)‘B 


where 


(X 

B  =  exp  - (n  -  c  +  i)J/  p(\|c)  dy,2. 


x  =  -r^- —  <C  9 - (n-c  +  i)^>  . 

^x,c.  C 


The  symbol  <->  means  that  the  expression  is  to  be  taken  as  zero 
if  the  contents  are  negative.  Subscripts  on  9' s  indicate  l.c.l.  (except 
for  9q)  and  subscripts  on  \  indicate  degrees  of  freedom. 


Bi  ovykv.  Igor  " F •  Labilit y  Theory  and  rn'Mc*  ’  Prentice-Hall.  Inc  1 961 


Mann.  N  R  Schafer.  P  E  and  Singpuns  alia  N  [<  'Method?  for  Stat i?tical  Analv?i?  of  Reliability  and  Life  Data  '  Wilev  L.tTi 


Hahn  and  5-hapiro  “.-tati'-t k a!  Model?  in  Engineering.'  Wiky  10f*7 


Bartholomew  D  J.  'The  sampling  Di?t  ribu»  ion  rf  an  Estimate  An?tng  in  Life  Testing’  Tec  hnomet  n<  -  \V.|  »,  *■>  \ 


mm®™® 


This  equation  is  a  weighted  sums  of  \'2(.  integrals.  This  is  quite  cumbersome  without 
the  aid  of  a  compiler.  especially  when  n  >  2.  It  is  obvious  that  even  for  a  sample  size 
of  two.  Bartholomew's  exact  method  is  quite  complicated.  Complications  and 
complexity-  u-ing  'Mribution  theory  leads,  in  most  cases,  to  the  use  of  asymptotic 
argument-  Dr  mb  ;< •nee  about  parameters.  However,  because  we  are  only  interested  in 
small  sample-,  a-ymptotie  results  are  not  applicable.  Therefore,  we  decided  to  compare 
Bartholomew  -  ex.-u-t  method  with  Epstein's  method  and  variations  of  Epstein's  method. 


II.  SIMULATION 

A  simulation  study  of  the  exponential  distribution  was  performed  with  0  =  1  and 
T*  =  1.  We  fir-t  chose  o  =  0.05  and  then  chose  o  =  0.10  while  letting  the  sample 
size.  n.  range  from  2  to  20.  For  each  sample  size,  2000  simulations  were  run. 

Uniform  random  numbers  were  used  to  generate  random  exponentially  di-tribute  I 
times.  After  each  exponential  time  was  generated  it  was  compared  to  T  =  I  If  the 
exponential  time  was  less  than  T  =  1.  then  it  was  considered  to  be  a  failure  occurring 
at  t;.  .After  n  of  these  times  were  generated  with  c  failures,  A  and  0  were  calculated. 
Lower  confidence  limits  for  0  were  then  calculated  using  Bartholomew's  exact  method. 
0exaft.  and  using  Epstein's  method  with  modifications  in  the  degrees  of  freedom  for  \". 
The  subscript  on  0  indicates  the  degrees  of  freedom  for  \2.  Thus  lower  confidence  limits 
U, .  +  ».  ^2c  +  l’  and  ^2c  were  calculated. 

The  lower  limit  on  reliability  was  calculated  using  the  computed  lower  confidence 
limit  for  9  in  the  following  equation 

-  X 

Rl(T)  =  e  Ml 

where 

T  =  Censored  time 

=  Computed  lower  confidence  limit. 


III.  DISCUSSION  OF  RESULTS 

For  those  simulations  where  o  =  .05,  we  would  expect  6 L  to  exceed  0  only  5rf  of 
the  time.  i.e..  I  *  (  i9l  >  9)  =  .05.  Since  there  were  2000  cases  for  each  sample  size,  we 
would  expect  100  of  our  to  exceed  0=1.  Table  1  shows  the  number  exceeding  1. 


t  Od»  neety  e  rb.n  T  0  and  0\  rrale  proportionally  ?o  if  T  —  60  hour?  9  —  300.  thru  6 ^  —  18  33901  If  T  and  0  v  ■  >!•  1 
bv  div.jing  by  f"  T  1  arid  0  -  0  SO  then  0 j  30b6i  which  i?  18  33901  -r  60  Thus  we  cbo?e  to  keep  T  1  f-'. r  ■  ■ I 

•x.impl—  a D d  '  mulaiio n« 


TABLE  1.  Simulation  Study  with  a  =  0.05 
Observed  Oases  where  >  9 
Expected  Number  =  100 


n 

n 

^  ex  act 

K 

^2  r  +  1 

02c  +  2 

2 

76 

0 

0 

0 

3 

95 

0 

0 

0 

4 

87 

261 

6 

0 

5 

102 

123 

118 

13 

6 

87 

162 

57 

47 

7 

104 

136 

108 

35 

8 

105 

156 

80 

69 

9 

111 

155 

110 

37 

10 

98 

134 

82 

54 

15 

98 

121 

88 

63 

20 

89 

106 

83 

63 

The  results  obtained  for  n  =  4.  5.  6.  7.  8,  9.  10.  15  and  20  for  the  9tx.in  are  typical 
of  what  one  expects.  However,  we  see  that  for  n  =  2,  the  exact  is  conservative.  B\ 
conservative  we  mean  that  the  lower  limit  tends  to  be  too  low,  since  in  less  than  5f7  of 
the  simulated  runs.  0exact  >  9  but  02c  ,  d2c  +  i  ,  and  92c  +  2  are  extremes,  i.e..  for  n  =  2 
and  3  there  are  no  0L  greater  than  0=1.  We  can  see  that  02c  +  2  *s  ver.v  conservative, 
with  02c  +  !  being  better.  For  02c,  we  get  more  than  100  lower  limits  exceeding  the  true 
value,  except  for  n  =  2  and  3. 

Similarly,  simulations  wore  run  with  a  =  .10.  Here  we  would  expect  200  cases  of 
the  0L  to  exceed  0.  Table  2  gives  the  actual  number  of  cases  for  each  sample  size. 
Again  we  see  that  92c  overestimates  the  number  of  lower  confidence  limits  oT  9{  which 
exceeds  the  true  value  of  0  which  equals  1  for  sample  sizes  except  n  =  2.  02i.+2  is  very 
conservative  and  02+1  is  less  conservative  (except  for  n  =  2  and  3). 


Another  observation  made  from  the  simulations  for  computed  values  of  0  was  that 
for  each  value  of  9.  the  lower  limit  on  9  tended  to  follow  the  pattern; 

^2c  +  2  <  ^2c  +  l  <  ^exact  <  Kc  ■  (•>) 

Examples  of  this  can  be  observed  for  n  =  10  and  20  by  examining  Figures  1  and  2. 
This  indicates  again  that  02r  +  2  and  02r  +  ,  are  conservative,  i.e.,  they  underestimate  (\. 
On  the  other  hand.  02c  overestimates  0 L  slightly. 

The  were  used  to  establish  a  lower  limit  on  reliability,  RL.  at  time  T.  Selected 
values  of  0  were  plotted  against  the  RL’s  for  the  four  methods  under  study.  The  plots 
appear  in  Figures  3  and  4  for  n  =  10  and  n  =  20,  respectively.  We  can  see  that  for 
small  values  of  0.  the  four  methods,  give  approximately  the  same  lower  limit  on 


TABLE  2.  Simulation  Study  with  a  =  0.10 
Observed  Cases  where  >  $ 
Expected  Number  =  200 


n 

^p.xrtct 

hr 

^2c+l 

$2  r  +  2 

2 

174 

0 

0 

0 

3 

190 

274 

0 

0 

4 

181 

252 

193 

2 

5 

192 

280 

126 

114 

6 

180 

263 

180 

67 

i 

188 

258 

152 

123 

8 

201 

279 

198 

110 

0 

193 

254 

163 

130 

10 

|  183 

225 

171 

112 

15 

1  19  1 

232 

167 

125 

20 

170 

203 

158 

117 

reliability.  For  larger  values  of  0.  we  can  see  that  R2f  lies  above  Rexart  everywhere,  with 


R2c  +  i  and  R2c  +  o  lying  generally  below  Rexac.t.  We  also  can  see  that  R2(.  +  ,  is  much 


closer  to  Rexact  than  the  other  two.  especially  for  n  =  20. 


For  each  sample  size,  the  maximum  absolute  difference  in  lower  reliability  estimates 
were  computed,  i.e.,  |  Rexaol  -  R2c  I-  I  Rcxact  -  R2c  +  ,  |  and  |  Rexact  -  R2c  +  2  |.  These 
differences  are  plotted  versus  sample  size  in  Figure  5.  We  can  see  that  the  differences 
were  larger  for  small  sample  sizes  (n  =  2  and  n  =  5)  and  generally  R2c  +  2  differed  the 


most  from  the  Rexact.  As  the  sample  size  got  larger  (n  =  10,  15  and  20),  all  the  reliabil¬ 


ity  estimates  converged  i.e.,  the  differences  converged  to  zero. 


Earlier  in  the  report  it  was  mentioned  that  one  alternative  to  this  confidence  limit 
estimate  problem  would  be  to  ignore  the  failure  times  and  simply  use  the  number  of 
failures  to  put  a  binomial  lower  confidence  limit  on  reliability.  To  show  how  conserva¬ 
tive  these  binomial  limits  are,  we  used  the  minimum  and  maximum  lower  reliability  esti¬ 
mates  obtained  from  Bartholomew's  exact  method  (2000  simulations;  n  =  10)  and  com¬ 
pared  them  to  the  binomial  lower  confidence  estimates.  These  comparisons  are  shown  in 
Figure  6  for  the  various  number  of  failures  observed.  We  can  see  that  the  binomial  lim¬ 
its  are  always  at  or  below  the  minimum  calculated  using  Bartholomew's  method.  Thus 
by  ignoring  the  failure  time  information,  we  are  lowering  our  estimate  of  reliability. 
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Figure  2.  Lower  95^  Confidence  Limits  on  9.  Exact  vs  Estimates  (n  =  20). 
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Figure  5.  Maximum  Absolute  Difference  in  Reliability  Lower  Confidence  Limits  for 
a  =  .05. 
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Figure  6.  Comparison  of  Binomial  with  Maximum  and  Minimal  Exact  Reliability  Low 
95%  Confidence  Limit. 


IV.  UNRESOLVED  FINDING 


During  the  simulation  it  was  noted  that  on  occasion  we  were  obtaining  estimate*  of 
dexact  that  were  greater  than  9.  Some  0exart’s  were  several  magnitudes  greater  than  0. 
Then  there  were  a  few  instances  where  computer  error  messages  were  obtained  instead 
of  an  estimate  of  0,.v., Looking  into  the  problem  we  noted  these  anomalies  were  occur¬ 
ring  when  6  approached  its  maximum  value. 


Take  for  instance  the  case  when  n  =  2,  c  =  1,  6  =  1,  T  =  1  and  a  =  .03 .  We  can 
obtain  a  closed  form  for  P  {0O  >  0  |  9}  —  q.  However,  this  is  not  a  continuous  func¬ 
tion:  there  are  three  ranges: 


For  [0  <  9  <  .5],  ft 


(t>) 


For  [.5  <  9  <  Ij.  a 


20  +  2 
0 


For  [1  <  0  <  2].  a 


(X) 


The  discontinuities  at  0  .  1  and  2  can  easily  be  seen  by  looking  at  the  maximum 
and  minimum  values  that  0  can  take  on.  Recall  that 

0  =  A  (<I) 

C 

V]  t|  +  (n  -  c)  T 

__  i=i _ 

c 

Let  (  be  a  very  small  time,  T  =  1.  and  n  =  2.  Then  for  one  failure  (c  =  1)  near  T. 
0  =  [(T  -  c)  4-  (n  1  )T] / 1  =2  c  is  the  maximum  value  that  6  can  assume.  F<»r  nc. 
failures  near  T. 

0  =  (t,  +  t2)/2  (101 

=  2(T  0/2 

=  I  f. 


Assume  one  failure  occurs  near  time  0,  0  =  [(0  4-  0  4-  T|/I  =  1  +  c  for  two  failures 
near  0.  0  =  [(0  +  2<)/2  =  0  +  f.  The  maximum  value  of  0  is  2  -  (.  This  indicates  that 
the  failure  occurred  only  t  time  before  the  censoring  time  T. 

To  study  the  problem,  we  selected  0  =  l.fl.j  and  started  at  0  .,  =  0.2  and  s< »1\ *  d 

for  o.  Then  0f.x:irt  was  increased  by  increnients  of  A  --  .2  until  0  t  readmd  20.  tie 
theoretical  maximum.  The  largest  n  obtained  while  solving  for  o  at  each  increment  was 
less  than  0.03  (See  Figure  7).  To  get  an  o=  0.03  for  0 =  l.fl.'j.  it  was  determined  ih:,i 
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0exarl  would  have  to  be  greater  than  500.  This  means  that  0exact  solved  in  the  equation 
would  he  much  larger  than  the  set  value.  Similarly,  when  n  =  3,  4,  and  5  there  were  ;t 
few  cases  where  0exart  >  6.  This  phenomena  did  not  occur  when  n  =  6,  7.  8,  10.  15 
and  20.  At  a  =  0.10.  this  also  occurred  for  n  =  2  thru  7.  Similarly  we  selected  0  = 
1.08  and  solved  for  o.  The  largest  a  obtained  was  less  than  0.015  (See  Figure  8).  ^  et 
we  have  not  been  able  to  explain  this  anomaly. 
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NOMENCLATURE 

maximum  likelihood  estimate 
lower  confidence  limit 
exponential  parameter 
m.l.e.  estimate  on  0 
l.c.l  on  0 

l.c.l.  on  0  using  Bartholomew's  exact  method 

l.c.l  on  0  using  Epstein's  method  -  subscript  indicates  d.f.  for  \' 

l.c.l  on  0  using  Epstein's  method  (modified)  -  subscript  indicates  d.f.  f>.r  \- 

l.c.l.  on  6  using  Epstein's  method  (modified)  -  subscript  indicates  d.f.  fur  \~ 

computed  l.c.l.  on  reliability  using  0L 

l.c.l.  on  reliability  using  0exart 

l.c.l  on  reliability  using  02c 

l.c.l.  on  reliability  using  02c  +  \ 

l.c.l.  on  reliability  using  02 f  +  2 
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